Attorney Docket No. 5074A-000057 
METHOD AND APPARATUS FOR PERSPECTIVE INVERSION 

FIELD OF THE INVENTION 
[0001] The present invention relates generally to surgical instrument 
navigation systems and, more particularly, to a navigation system that provides 
perspective inversion of the surgical instrument. 

BACKGROUND OF THE INVENTION 
[0002] Modern diagnostic medicine has benefited significantly from 
radiology. Radiation, such as x-rays, may be used to generate images of internal 
body structures. In general, radiation is emanated towards a patient's body and 
absorbed in varying amounts by tissues in the body. An x-ray image is then 
created based on the relative differences of detected radiation passing through 
the patients' body. 

[0003] Surgical navigation guidance can provide a tool for helping the 
physician perform surgery. One known technique involves tracking position in 
real-time of a surgical instrument in the patient's anatomy as it is represented by 
an x-ray image. The virtual representation of the surgical instrument is a three- 
dimensional object superimposed onto the two-dimensional image of the patient. 
Thus, the three-dimensional representation appears to be directed into or out of 
the two-dimensional image of the patient. An exemplary surgical navigation 
guidance system is disclosed in U.S. Application Serial No. 09/274,972 filed on 
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March 23, 1999 which is assigned to the assignee of the present invention and 
incorporated herein by reference. 

[0004] When an image is acquired, it is acquired from a certain 
perspective or point-of-view. In the case of a C-arm imaging device, the 
perspective is determined by the orientation of the C-Arm around the patient. 
Specifically, the perspective is along the line connecting the image source and 
the image receiver. If the surgeon navigates the surgical instrument from the 
position of the image receiver, the perspective of the virtual representation of the 
instrument will match the surgeon's perspective of the actual instrument. 
However, if the surgeon navigates from the position of the radiation source, the 
perspective of the virtual representation of the instrument will appear "flipped" 
from the surgeon's perspective of the actual instrument. 

[0005] Therefore, it is desirable to provide a surgical navigation system 
that allows the surgeon to invert or "flip" the three-dimensional perspective of the 
instrument to match their perspective of the actual instrument. 

SUMMARY OF THE INVENTION 
[0006] In accordance with the present invention, a surgical instrument 
navigation system is provided that allows a surgeon to invert the three- 
dimensional perspective of the instrument to match their perspective of the actual 
instrument. The surgical instrument navigation system includes: a surgical 
instrument; an imaging device that is operable to capture image data 
representative of a patient; a tracking subsystem that is operable to capture in 
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real-time position data indicative of the position of the surgical instrument; and a 
data processor adapted to receive the image data from the imaging device and 
the position data from the tracking subsystem. The data processor is operable to 
generate a three-dimensional representation of the surgical instrument as it 
would visually appear from either of at least two different perspectives and to 
overlay the representation of the surgical instrument onto the image data of the 
patient. The navigation system further includes a display that is operable to 
display the representation of the surgical instrument superimposed onto the 
image data of the patient. 

[0007] For a more complete understanding of the invention, reference 
may be had to the following specification and to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0008] Figure 1 is a diagram of a surgical instrument navigation 
system in accordance with present invention; 

[0009] Figure 2 is a diagram of an ideal and distorted image that may 
be captured by the surgical navigation system; 

[0010] Figures 3A and 3B illustrates the projective transformation 
process employed by the surgical navigation system; 

[0011] Figure 4 is a flowchart depicting the operation of the surgical 
navigation system; 

[0012] Figures 5A and 5B illustrate graphical representations of the 
surgical instrument superimposed onto a two-dimensional image of the patient; 
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[0013] Figure 6 illustrates an exemplary graphical user interface of the 
surgical instrument navigation system; and 

[0014] Figure 7 is a flowchart depicting how perspective inversion is 
incorporated into the operation of the surgical instrument navigation system in 
accordance with the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0015] Figure 1 is a diagram of an exemplary surgical instrument 
navigation system. The primary component of the surgical instrument navigation 
system is a fluoroscopic imaging device 100. The fluoroscopic imaging device 
100 generally includes a C-arm 103, x-ray source 104, x-ray receiving section 
105, a calibration and tracking target 106, and radiation sensors 107. Calibration 
and tracking target 106 includes infrared reflectors (or alternatively infrared 
emitters) 109 and calibration markers 111. C-arm control computer 115 allows a 
physician to control the operation of imaging device 100, such as setting imaging 
parameters. One appropriate implementation of imaging device 100 is the 
"Series 9600 Mobile Digital Imaging System," from OEC Medical Systems, Inc., 
of Salt Lake City, Utah. It should be noted that calibration and tracking target 
106 and radiation sensors 107 are typically not included in the Series 9600 
Mobile Digital Imaging System; otherwise the "Series 9600 Mobile Digital 
Imaging System" is similar to imaging system 100. 

[0016] In operation, x-ray source 104 generates x-rays that propagate 
through patient 110 and calibration target 106, and into x-ray receiving section 
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105. Receiving section 105 generates an image representing the intensities of 
the received x-rays. Typically, receiving section 105 comprises an image 
intensifier that converts the x-rays to visible light and a charge coupled device 
(CCD) video camera that converts the visible light to digital images. Receiving 
section 105 may also be a device that converts x-rays directly to digital images, 
thus potentially avoiding distortion introduced by first converting to visible light. 

[0017] Fluoroscopic images taken by imaging device 100 are 
transmitted to computer 115, where they may further be forwarded to computer 
120. Computer 120 provides facilities for displaying (on monitor 121), saving, 
digitally manipulating, or printing a hard copy of the received images. Three- 
dimensional images, such as pre-acquired patient specific CT/MR data set 124 
or a three-dimensional atlas data set 126 may also be manipulated by computer 
120 and displayed by monitor 121. Images, instead of or in addition to being 
displayed on monitor 121, may also be displayed to the physician through a 
heads-up-display. 

[0018] Although computers 115 and 120 are shown as two separate 
computers, they alternatively could be variously implemented as multiple 
computers or as a single computer that performs the functions performed by 
computers 115 and 120. In this case, the single computer would receive input 
from both C-arm imager 100 and tracking sensor 130. 

[0019] Radiation sensors 107 sense the presence of radiation, which is 
used to determine whether or not imaging device 100 is actively imaging. The 
result of their detection is transmitted to processing computer 120. Alternatively, 
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a person may manually indicate when device 100 is actively imaging or this 
function can be built into x-ray source 104, x-ray receiving section 105, or control 
computer 115. 

[0020] In operation, the patient is positioned between the x-ray source 
104 and the x-ray receiving section 105. In response to an operator's command 
input at control computer 1 15, x-rays emanate from source 104 and pass through 
patient 110, calibration target 106, and into receiving section 105 which 
generates a two-dimensional image of the patient. 

[0021] C-arm 103 is capable of rotating relative to patient 110, thereby 
allowing images of patient 1 10 to be taken from multiple directions. For example, 
the physician may rotate C-arm 103 in the direction of arrows 108 or about the 
long axis of the patient. Each of these directions of movement involves rotation 
about a mechanical axis of the C-arm. In this example, the long axis of the 
patient is aligned with the mechanical axis of the C-arm. 

[0022] Raw images generated by receiving section 105 tend to suffer 
from undesirable distortion caused by a number of factors, including inherent 
image distortion in the image intensifier and external electromagnetic fields. 
Drawings representing ideal and distorted images are shown in Figure 2. 
Checkerboard 202 represents the ideal image of a checkerboard shaped object. 
The image taken by receiving section 105, however, can suffer significant 
distortion, as illustrated by distorted image 204. 

[0023] The image formation process in a system such as fluoroscopic 
C-arm imager 100 is governed by a geometric projective transformation which 
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maps lines in the fluoroscope's field of view to points in the image (i.e., within the 
x-ray receiving section 105). This concept is illustrated in Figures 3A and 3B. 
Image 300 (and any image generated by the fluoroscope) is composed of 
discrete picture elements (pixels), an example of which is labeled as 302. Every 
pixel within image 300 has a corresponding three-dimensional line in the 
fluoroscope's field of view. For example, the line corresponding to pixel 302 is 
labeled as 304. The complete mapping between image pixels and corresponding 
lines governs projection of objects within the field of view into the image. The 
intensity value at pixel 302 is determined by the densities of the object elements 
(i.e., portions of a patient's anatomy, operating room table, etc.) intersected by 
the line 304. For the purposes of computer assisted navigational guidance, it is 
necessary to estimate the projective transformation which maps lines in the field 
of view to pixels in the image, and vice versa. Geometric projective 
transformation is well known in the art. 

[0024] Intrinsic calibration, which is the process of correcting image 
distortion in a received image and establishing the projective transformation for 
that image, involves placing "calibration markers" in the path of the x-ray, where 
a calibration marker is an object opaque or semi-opaque to x-rays. Calibration 
markers 111 are rigidly arranged in predetermined patterns in one or more 
planes in the path of the x-rays and are visible in the recorded images. Tracking 
targets, such as emitters or reflectors 109, are fixed in a known position relative 
to calibration markers 111. 
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[0025] Because the true relative position of the calibration markers 1 1 1 
in the recorded images are known, computer 120 is able to calculate an amount 
of distortion at each pixel in the image (where a pixel is a single point in the 
image). Accordingly, computer 120 can digitally compensate for the distortion in 
the image and generate a distortion-free, or at least a distortion improved image. 
Alternatively, distortion may be left in the image, and subsequent operations on 
the image, such as superimposing an iconic representation of a surgical 
instrument on the image (described in more detail below), may be distorted to 
match the image distortion determined by the calibration markers. The 
calibration markers can also be used to estimate the geometric perspective 
transformation, since the position of these markers are known with respect to the 
tracking target emitters or reflectors 109 and ultimately with respect to tracking 
sensor 130. A more detailed explanation of methods for performing intrinsic 
calibration is described in the references B. Schuele et al., "Correction of Image 
Intensifier Distortion for Three-Dimensional Reconstruction," presented at SPIE 
Medical Imaging 1995, San Diego, CA, 1995 and G. Champleboux et al., 
"Accurate Calibration of Cameras and Range Imaging Sensors: the NPBS 
Method," Proceedings of the 1992 IEEE International Conference on Robotics 
and Automation, Nice, France, May 1992, and U.S. Application Serial Number 
09/106,109, filed on June 29, 1998 by the present assignee, the contents of 
which are hereby incorporated by reference. 

[0026] Calibration and tracking target 106 may be attached to x-ray 
receiving section 105 of the C-arm. Alternately, the target 106 can be 
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mechanically independent of the C-arm, in which case it should be positioned 
such that the included calibration markers 111 are visible in each fluoroscopic 
image to be used in navigational guidance. Element 106 serves two functions. 
The first, as described above, is holding calibration markers 1 1 1 used in intrinsic 
calibration. The second function, which is described in more detail below, is 
holding infrared emitters or reflectors 109, which act as a tracking target for 
tracking sensor 130. 

[0027] Tracking sensor 130 is a real-time infrared tracking sensor 
linked to computer 120. Specially constructed surgical instruments and other 
markers in the field of tracking sensor 130 can be detected and located in three- 
dimensional space. For example, a surgical instrument 140, such as a drill, is 
embedded with infrared emitters or reflectors 141 on its handle. Tracking sensor 
130 detects the presence and location of infrared emitters or reflectors 141. 
Because the relative spatial locations of the emitters or reflectors in instrument 
140 are known a priori, tracking sensor 130 and computer 120 are able to locate 
instrument 140 in three-dimensional space using well known mathematical 
transformations. Instead of using infrared tracking sensor 130 and 
corresponding infrared emitters or reflectors, other types of positional location 
devices which are known in the art may be used. For example, positional 
location devices based on magnetic fields, sonic emissions, or radio waves are 
also within the scope of the present invention. 

[0028] Reference frame marker 150, like surgical instrument 140, is 
embedded with infrared emitters or reflectors, labeled 151. As with instrument 
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140, tracking sensor 130 similarly detects the spatial location of 
emitters/reflectors 151, through which tracking sensor 130 and computer 120 
determine the three-dimensional position of dynamic reference frame marker 
150. The determination of the three-dimensional position of an object relative to 
a patient is known in the art, and is discussed, for example, in the following 
references, each of which is hereby incorporated by reference: PCT Publication 
WO 96/11624 to Bucholz et al., published April 25, 1996; U.S. Patent No. 
5,384,454 to Bucholz; U.S. Patent No. 5,851,183 to Bucholz; and U.S. Patent 
No. 5,871 ,445 to Bucholz. 

[0029] During an operation, dynamic reference frame marker 150 is 
attached in a fixed position relative to the portion of the patient to be operated on. 
For example, when inserting a screw into the spine of patient 110, dynamic 
reference frame marker 150 may be physically attached to a portion of the spine 
of the patient. Because dynamic reference frame 150 is in a fixed position 
relative to the patient anatomy, and instrument 140 can be accurately located in 
three dimensional space relative to dynamic reference frame 150, instrument 140 
can also be located relative to the patient's anatomy. 

[0030] As discussed above, calibration and tracking target 106 also 
includes infrared emitters or reflectors 109 similar to those in instrument 140 or 
dynamic reference frame 150. Accordingly, tracking sensor 130 and computer 
120 may determine the three-dimensional position of calibration target 106 
relative to instrument 140 and/or dynamic reference frame 150 and thus the 
patient position. 
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[0031] In general, the imaging system assists physicians performing 
surgery by displaying real-time or pre-acquired images, such as fluoroscopic x- 
ray images, of the patient 110 on display 121. Representations of surgical 
instruments 140 are overlaid on pre-acquired fluoroscopic images of patient 110 
based on the position of the instruments determined by tracking sensor 130. In 
this manner, the physician is able to see the location of the instrument relative to 
the patient's anatomy, without the need to acquire real-time fluoroscopic images, 
thereby greatly reducing radiation exposure to the patient and to the surgical 
team. "Pre-acquired," as used herein, is not intended to imply any required 
minimum duration between receipt of the x-ray signals and displaying the 
corresponding image. Momentarily storing the corresponding digital signal in 
computer memory while displaying the fluoroscopic image constitutes pre- 
acquiring the image. 

[0032] Figure 4 is a flowchart depicting the operation of the surgical 
navigation system. The physician begins by acquiring one or more fluoroscopic 
x-ray images of patient 110 using imager 100 (step 400). As previously 
mentioned, acquiring an x-ray image triggers radiation sensors 107, which 
informs computer 120 of the beginning and end of the radiation cycle used to 
generate the image. For a fluoroscopic x-ray image acquired with imager 100 to 
be useable for navigational guidance, imager 100, when acquiring the image, 
should be stationary with respect to patient 110. If C-arm 103 or patient 110 is 
moving during image acquisition, the position of the fluoroscope will not be 
accurately determined relative to the patient's reference frame. Thus, it is 
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important that the recorded position of imager 1 00 reflects the true position of the 
imager at the time of image acquisition. If imager 100 moves during the image 
acquisition process, or if imager 100 moves after image acquisition but before its 
position is recorded, the calibration will be erroneous, thereby resulting in 
incorrect graphical overlays. To prevent this type of erroneous image, computer 
120 may examine the position information from tracking sensor 130 while 
radiation sensors 107 are signaling radiation detection. If the calibration and 
tracking target 106 moves relative to dynamic reference frame 150 during image 
acquisition, this image is marked as erroneous (Steps 401 and 402). 

[0033] At the end of the radiation cycle, computer 120 retrieves the 
acquired image from C-arm control computer 115 and retrieves the location 
information of target marker 106 and dynamic reference frame 150 from tracking 
sensor 130. Computer 120 calibrates the acquired image, as described above, 
to learn its projective transformation and optionally to correct distortion in the 
image, (step 403), and then stores the image along with its positional information 
(step 404). The process of steps 400-404 is repeated for each image that is to 
be acquired (step 405). 

[0034] Because the acquired images are stored with the positional 
information of the calibration and tracking target 106 and dynamic reference 
frame 150, the position of C-arm 103, x-ray source 104, and receiving section 
105 for each image, relative to patient 110, can be computed based upon the 
projective transformation identified in the calibration process. During surgery, 
tracking sensor 130 and computer 120 detect the position of instrument 140 
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relative to dynamic reference frame 150, and hence relative to patient 110. With 
this information, computer 120 dynamically calculates, in real-time, the projection 
of instrument 140 into each fluoroscopic image as the instrument is moved by the 
physician. A graphical representation of instrument 140 may then be overlaid on 
the fluoroscopic images (step 406). The graphical representation of instrument 
140 is an iconic representation of where the actual surgical instrument would 
appear within the acquired fluoroscopic x-ray image if imager 100 was 
continuously acquiring new images from the same view as the original image. 
There is no theoretical limit to the number of fluoroscopic images on which the 
graphical representations of instrument 140 may be simultaneously overlaid. 

[0035] The graphical representation of the surgical instrument is a 
three-dimensional object superimposed onto a two-dimensional image of the 
patient. The three-dimensional representation of the instrument may appear to 
be directed into or out of the two-dimensional image as shown in Figures 5A and 
5B. In Figure 5A, the tip 502 of the instrument 504 and the projected length 
appear to be directed into the image. Conversely, in Figure 5B, the tip 502 of the 
instrument 504 and the projected length appear to be coming out of the image. 

[0036] When an image is acquired, it is acquired from a certain 
perspective or point-of-view. In the case of a C-arm imaging device 100, the 
perspective is determined by the orientation of the C-Arm 1 03 around the patient 
110. Specifically, the perspective is along the line connecting the image source 
104 and the image receiver section 105. If the surgeon navigates the surgical 
instrument from the position of the image receiver section 105, the perspective of 
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the virtual representation of the instrument will match the surgeon's perspective 
of the actual instrument. However, if the surgeon navigates from the position of 
the image source 104, the perspective of the virtual representation of the 
instrument will appear "flipped" from the surgeon's perspective of the actual 
instrument. 

[0037] In accordance with the present invention, the surgical 
instrument navigation system described above has been enhanced to allow a 
surgeon to invert the graphical representation of the instrument to match their 
perspective of the actual instrument. In a preferred embodiment, the navigation 
system provides two possible perspectives: positive (+) or negative (-). The 
positive state renders the instrument from the perspective of the image receiver 
section 105; whereas the negative state renders the instrument from the 
perspective of the image source 104. It is envisioned that either state may be 
designated the default state. It is further envisioned that more than two 
perspectives may be available for selection by the surgeon. 

[0038] Referring to Figure 6, the perspective of the instrument is 
selectable using a touch screen operable button 601 provided on the graphical 
user interface of the navigation system. One skilled in the art will readily 
recognize that rendering a particular perspective of the instrument does not affect 
the profile of the instrument or the location of the instrument on the image. The 
perspective selection only affects the internal contours that give the instrument 
the appearance into or out of the image as shown in Figures 5A and 5B. 
Although a touch screen operable button is presently preferred, it is envisioned 
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that other techniques for selecting the perspective of the instrument, such as a 
foot pedal or other switching device in close proximity to the surgeon, are also 
within the scope of the present invention. 

[0039] A more detailed description of how perspective inversion is 
incorporated into the operation of the surgical instrument navigation system is 
provided in conjunction with Figure 7. As noted above, the projection of the 
instrument into the fluoroscopic image is calculated in real-time as the instrument 
is moved by the surgeon. 

[0040] To do so, the tracking sensor 130, in conjunction with the 
computer 120, detects the position of the instrument 140 at step 702 relative to 
the dynamic reference frame 150, and thus relative to the patient 110. The 
tracking sensor 130, in conjunction with the computer 120, also determines the 
position of the tracking target 106 at step 704 relative to the dynamic reference 
frame 150. Based this position data, the computer 120 can determine the 
position of the instrument 140 relative to the tracking target 106 at step 706, and 
calibrate the position of the instrument relative to the image plane of the 
fluoroscopic images at step 708. 

[0041] Prior to rending the image, the navigation system accounts for 
the various user settings 714, including instrument perspective. The selected 
perspective setting 714 is input into the computer 120 at step 710 which in turn 
provides corresponding input to the graphic rendering software. One skilled in 
the art will readily recognize that other user settings (e.g., zoom, rotate, ect.) may 
be accounted for by the navigation system. 
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[0042] Lastly, the fluoroscopic image is rendered by the navigation 
system at step 712. Specifically, the three-dimensional representation of the 
surgical instrument is rendered from the perspective input by an operator of the 
navigation system. The representation of the instrument is then superimposed 
over the previously calibrated image data for the patient. In this way, the 
perspective of the displayed instrument matches the surgeon's perspective of the 
actual instrument. As noted above, the representation of the surgical instrument 
is tracked in real-time as it is moved by the surgeon. 

[0043] While the invention has been described in its presently preferred 
form, it will be understood that the invention is capable of modification without 
departing from the spirit of the invention as set forth in the appended claims. 
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